Renewal Theory for Markov Chains on the Real Line

Renewal Theorems for Markov Chains

Taylor Expansion for the Entropy Rate of Hidden Markov Chains

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

i?-THEORY FOR MARKOV CHAINS

If {Xn} is a discrete-time ^-irreducible Markov chain on a measure space (#*, 3?) [4; p. 4], with «-step transition probabilities P"(x, A), it has been shown in [5] that there exists a subset ^R of 3F with the property that, for every Ae^R and 0-almost all x € W, the power series J^P(x, A)z" have the same radius of convergence R. Moreover, there is a countable partition of 2£ all of whose eleme...

The Rate of Rényi Entropy for Irreducible Markov Chains

In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.

Renewal theory and computable convergence rates for geometrically ergodic Markov chains

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and Tweedie, and from estimates using coupling, although we start from essentially the same assumptions of a drift condition towards a “small set”. The estimates sh...